Merge Generator

The Merge generator combines parameters produced by the base (first) generator with matching parameter sets produced by subsequent generators. A matching parameter set has the same values for the configured merge keys. Non-matching parameter sets are discarded. Override precedence is bottom-to-top: the values from a matching parameter set produced by generator 3 will take precedence over the values from the corresponding parameter set produced by generator 2.

Using a Merge generator is appropriate when a subset of parameter sets require overriding.

Example: Base Cluster generator + override Cluster generator + List generator

As an example, imagine that we have two clusters:

  • A staging cluster (at https://1.2.3.4)
  • A production cluster (at https://2.4.6.8)
  1. apiVersion: argoproj.io/v1alpha1
  2. kind: ApplicationSet
  3. metadata:
  4.   name: cluster-git
  5. spec:
  6.   goTemplate: true
  7.   goTemplateOptions: ["missingkey=error"]
  8.   generators:
  9.     # merge 'parent' generator
  10.     - merge:
  11.         mergeKeys:
  12.           - server
  13.         generators:
  14.           - clusters:
  15.               values:
  16.                 kafka: 'true'
  17.                 redis: 'false'
  18.           # For clusters with a specific label, enable Kafka.
  19.           - clusters:
  20.               selector:
  21.                 matchLabels:
  22.                   use-kafka: 'false'
  23.               values:
  24.                 kafka: 'false'
  25.           # For a specific cluster, enable Redis.
  26.           - list:
  27.               elements: 
  28.                 - server: https://2.4.6.8
  29.                   values.redis: 'true'
  30.   template:
  31.     metadata:
  32.       name: '{{.name}}'
  33.     spec:
  34.       project: '{{index .metadata.labels "environment"}}'
  35.       source:
  36.         repoURL: https://github.com/argoproj/argo-cd.git
  37.         targetRevision: HEAD
  38.         path: app
  39.         helm:
  40.           parameters:
  41.             - name: kafka
  42.               value: '{{.values.kafka}}'
  43.             - name: redis
  44.               value: '{{.values.redis}}'
  45.       destination:
  46.         server: '{{.server}}'
  47.         namespace: default

The base Cluster generator scans the set of clusters defined in Argo CD, finds the staging and production cluster secrets, and produces two corresponding sets of parameters:

  1. - name: staging
  2.   server: https://1.2.3.4
  3.   values.kafka: 'true'
  4.   values.redis: 'false'
  6. - name: production
  7.   server: https://2.4.6.8
  8.   values.kafka: 'true'
  9.   values.redis: 'false'

The override Cluster generator scans the set of clusters defined in Argo CD, finds the staging cluster secret (which has the required label), and produces the following parameters:

  1. - name: staging
  2.   server: https://1.2.3.4
  3.   values.kafka: 'false'

When merged with the base generator’s parameters, the values.kafka value for the staging cluster is set to 'false'.

  1. - name: staging
  2.   server: https://1.2.3.4
  3.   values.kafka: 'false'
  4.   values.redis: 'false'
  6. - name: production
  7.   server: https://2.4.6.8
  8.   values.kafka: 'true'
  9.   values.redis: 'false'

Finally, the List cluster generates a single set of parameters:

  1. - server: https://2.4.6.8
  2.   values.redis: 'true'

When merged with the updated base parameters, the values.redis value for the production cluster is set to 'true'. This is the merge generator’s final output:

  1. - name: staging
  2.   server: https://1.2.3.4
  3.   values.kafka: 'false'
  4.   values.redis: 'false'
  6. - name: production
  7.   server: https://2.4.6.8
  8.   values.kafka: 'true'
  9.   values.redis: 'true'

Example: Use value interpolation in merge

Some generators support additional values and interpolating from generated variables to selected values. This can be used to teach the merge generator which generated variables to use to combine different generators.

The following example combines discovered clusters and a git repository by cluster labels and the branch name:

  1. apiVersion: argoproj.io/v1alpha1
  2. kind: ApplicationSet
  3. metadata:
  4.   name: cluster-git
  5. spec:
  6.   goTemplate: true
  7.   goTemplateOptions: ["missingkey=error"]
  8.   generators:
  9.     # merge 'parent' generator:
  10.     # Use the selector set by both child generators to combine them.
  11.     - merge:
  12.         mergeKeys:
  13.           # Note that this would not work with goTemplate enabled,
  14.           # nested merge keys are not supported there.
  15.           - values.selector
  16.         generators:
  17.           # Assuming, all configured clusters have a label for their location:
  18.           # Set the selector to this location.
  19.           - clusters:
  20.               values:
  21.                 selector: '{{index .metadata.labels "location"}}'
  22.           # The git repo may have different directories which correspond to the
  23.           # cluster locations, using these as a selector.
  24.           - git:
  25.               repoURL: https://github.com/argoproj/argocd-example-apps/
  26.               revision: HEAD
  27.               directories:
  28.               - path: '*'
  29.               values:
  30.                 selector: '{{.path.path}}'
  31.   template:
  32.     metadata:
  33.       name: '{{.name}}'
  34.     spec:
  35.       project: '{{index .metadata.labels "environment"}}'
  36.       source:
  37.         repoURL: https://github.com/argoproj/argocd-example-apps/
  38.         # The cluster values field for each generator will be substituted here:
  39.         targetRevision: HEAD
  40.         path: '{{.path.path}}'
  41.       destination:
  42.         server: '{{.server}}'
  43.         namespace: default

Assuming a cluster named germany01 with the label metadata.labels.location=Germany and a git repository containing a directory called Germany, this could combine to values as follows:

  1.   # From the cluster generator
  2. - name: germany01
  3.   server: https://1.2.3.4
  4.   # From the git generator
  5.   path: Germany
  6.   # Combining selector with the merge generator
  7.   values.selector: 'Germany'
  8.   # More values from cluster & git generator
  9.   # […]

Restrictions

  1. You should specify only a single generator per array entry. This is not valid:

    1. - merge:
    2.     generators:
    3.     - list: # (...)
    4.       git: # (...)
    • While this will be accepted by Kubernetes API validation, the controller will report an error on generation. Each generator should be specified in a separate array element, as in the examples above.

  2. The Merge generator does not support template overrides specified on child generators. This template will not be processed:

    1. - merge:
    2.     generators:
    3.       - list:
    4.           elements:
    5.             - # (...)
    6.           template: { } # Not processed

  3. Combination-type generators (Matrix or Merge) can only be nested once. For example, this will not work:

    1. - merge:
    2.     generators:
    3.       - merge:
    4.           generators:
    5.             - merge:  # This third level is invalid.
    6.                 generators:
    7.                   - list:
    8.                       elements:
    9.                         - # (...)

  4. Merging on nested values while using goTemplate: true is currently not supported, this will not work

    1. spec:
    2.   goTemplate: true
    3.   generators:
    4.   - merge:
    5.       mergeKeys:
    6.         - values.merge

  5. When using a Merge generator nested inside another Matrix or Merge generator, Post Selectors for this nested generator’s generators will only be applied when enabled via spec.applyNestedSelectors.

    1. - merge:
    2.     generators:
    3.       - merge:
    4.           generators:
    5.             - list
    6.                 elements:
    7.                   - # (...)
    8.               selector: { } # Only applied when applyNestedSelectors is true